{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 多位数字识别\n",
    "\n",
    "应用背景：将建筑物添加到谷歌地图中，街景车拍摄建筑物，同时记录各张照片中建筑物相关的GPS坐标，通过机器识别每张图片上的门牌号码，方便谷歌地图数据库在正确的位置添加该建筑地址。\n",
    "\n",
    "多位数字识别问题是一种特殊类型的序列识别（sequence recognition），其特殊之处在于：序列的长度是有界的，极少出现门牌号包括5位以上的数字。本例中假设门牌号最长5位。\n",
    "\n",
    "传统做法：将操作步骤分为定位(localization)、分割(segmentation)、识别(recognition)三个阶段，[Goodfellow,2014](https://arxiv.org/abs/1312.6082)提出一种基于深度卷积神经网络的三个阶段整合的方案，这一方案也是本例实践的主要指导。本例代码主要参考[potterhsu](https://github.com/potterhsu/SVHNClassifier)、[thomalm](https://github.com/thomalm/svhn-multi-digit)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import h5py\n",
    "import random\n",
    "from PIL import Image\n",
    "import tensorflow as tf\n",
    "from datetime import datetime\n",
    "import time\n",
    "import json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据处理方面，由于SVHN数据集某些数字裁剪的太紧，导致地址数字被裁剪操作剪除。因此需要将裁剪区域范围扩大：\n",
    "> expand this bounding box by 30% in both the x and the y direction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def preProcess(image, bbox_left, bbox_top, bbox_width, bbox_height):\n",
    "        cropped_left, cropped_top, cropped_width, cropped_height = (int(round(bbox_left - 0.15 * bbox_width)),\n",
    "                                                                    int(round(bbox_top - 0.15 * bbox_height)),\n",
    "                                                                    int(round(bbox_width * 1.3)),\n",
    "                                                                    int(round(bbox_height * 1.3)))\n",
    "        image = image.crop([cropped_left, cropped_top, cropped_left + cropped_width, cropped_top + cropped_height])\n",
    "        image = image.resize([64, 64])\n",
    "        return image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为增加数据样本量，将原本一张64$*$64像素的图片随机裁剪为若干54$*$54像素的图片：\n",
    "> crop a 54$*$54 pixel image from a random location within the 64$*$64 pixel image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Donkey(object):\n",
    "    @staticmethod\n",
    "    def _preprocess(image):\n",
    "        image = tf.image.convert_image_dtype(image, dtype=tf.float32)\n",
    "        image = tf.multiply(tf.subtract(image, 0.5), 2)\n",
    "        image = tf.reshape(image, [64, 64, 3])\n",
    "        image = tf.random_crop(image, [54, 54, 3])\n",
    "        return image\n",
    "\n",
    "    @staticmethod\n",
    "    def _read_and_decode(filename_queue):\n",
    "        reader = tf.TFRecordReader()\n",
    "        _, serialized_example = reader.read(filename_queue)\n",
    "        features = tf.parse_single_example(\n",
    "            serialized_example,\n",
    "            features={\n",
    "                'image': tf.FixedLenFeature([], tf.string),\n",
    "                'length': tf.FixedLenFeature([], tf.int64),\n",
    "                'digits': tf.FixedLenFeature([5], tf.int64)\n",
    "            })\n",
    "\n",
    "        image = Donkey._preprocess(tf.decode_raw(features['image'], tf.uint8))\n",
    "        length = tf.cast(features['length'], tf.int32)\n",
    "        digits = tf.cast(features['digits'], tf.int32)\n",
    "        return image, length, digits\n",
    "\n",
    "    @staticmethod\n",
    "    def build_batch(path_to_tfrecords_file, num_examples, batch_size, shuffled):\n",
    "        assert tf.gfile.Exists(path_to_tfrecords_file), '%s not found' % path_to_tfrecords_file\n",
    "\n",
    "        filename_queue = tf.train.string_input_producer([path_to_tfrecords_file], num_epochs=None)\n",
    "        image, length, digits = Donkey._read_and_decode(filename_queue)\n",
    "\n",
    "        min_queue_examples = int(0.4 * num_examples)\n",
    "        if shuffled:\n",
    "            image_batch, length_batch, digits_batch = tf.train.shuffle_batch([image, length, digits],\n",
    "                                                                             batch_size=batch_size,\n",
    "                                                                             num_threads=2,\n",
    "                                                                             capacity=min_queue_examples + 3 * batch_size,\n",
    "                                                                             min_after_dequeue=min_queue_examples)\n",
    "        else:\n",
    "            image_batch, length_batch, digits_batch = tf.train.batch([image, length, digits],\n",
    "                                                                     batch_size=batch_size,\n",
    "                                                                     num_threads=2,\n",
    "                                                                     capacity=min_queue_examples + 3 * batch_size)\n",
    "        return image_batch, length_batch, digits_batch\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "模型结构及具体参数沿用Goodfellow的模型方案：\n",
    "> - Our best architecture consists of eight convolutional hidden layers, one locally connected hidden layer, and two densely connected hidden layers. All connections are feedforward and go from one layer to the next (no skip connections).\n",
    "> - The number of units at each spatial location in each layer is [48, 64, 128, 160] for the first four layers and 192 for all other locally connected layers. The fully connected layers contain 3,072 units each. \n",
    "> - Each convolutional layer includes max pooling and subtractive normalization. The max pooling window size is 2$*$2. \n",
    "> - All convolutions use zero padding on the input to preserve representation size.\n",
    "> - All convolution kernels were of size 5$*$5. We trained with dropout applied to all hidden layers but not the input.\n",
    "\n",
    "模型机构如图所示：\n",
    "![model-graph](../img/model-graph.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Model(object):\n",
    "\n",
    "    @staticmethod\n",
    "    def inference(x, drop_rate):\n",
    "        with tf.variable_scope('hidden1'):\n",
    "            conv = tf.layers.conv2d(x, filters=48, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=2, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden1 = dropout\n",
    "\n",
    "        with tf.variable_scope('hidden2'):\n",
    "            conv = tf.layers.conv2d(hidden1, filters=64, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=1, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden2 = dropout\n",
    "\n",
    "        with tf.variable_scope('hidden3'):\n",
    "            conv = tf.layers.conv2d(hidden2, filters=128, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=2, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden3 = dropout\n",
    "\n",
    "        with tf.variable_scope('hidden4'):\n",
    "            conv = tf.layers.conv2d(hidden3, filters=160, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=1, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden4 = dropout\n",
    "\n",
    "        with tf.variable_scope('hidden5'):\n",
    "            conv = tf.layers.conv2d(hidden4, filters=192, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=2, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden5 = dropout\n",
    "\n",
    "        with tf.variable_scope('hidden6'):\n",
    "            conv = tf.layers.conv2d(hidden5, filters=192, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=1, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden6 = dropout\n",
    "\n",
    "        with tf.variable_scope('hidden7'):\n",
    "            conv = tf.layers.conv2d(hidden6, filters=192, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=2, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden7 = dropout\n",
    "\n",
    "        with tf.variable_scope('hidden8'):\n",
    "            conv = tf.layers.conv2d(hidden7, filters=192, kernel_size=[5, 5], padding='same')\n",
    "            norm = tf.layers.batch_normalization(conv)\n",
    "            activation = tf.nn.relu(norm)\n",
    "            pool = tf.layers.max_pooling2d(activation, pool_size=[2, 2], strides=1, padding='same')\n",
    "            dropout = tf.layers.dropout(pool, rate=drop_rate)\n",
    "            hidden8 = dropout\n",
    "\n",
    "        flatten = tf.reshape(hidden8, [-1, 4 * 4 * 192])\n",
    "\n",
    "        with tf.variable_scope('hidden9'):\n",
    "            dense = tf.layers.dense(flatten, units=3072, activation=tf.nn.relu)\n",
    "            hidden9 = dense\n",
    "\n",
    "        with tf.variable_scope('hidden10'):\n",
    "            dense = tf.layers.dense(hidden9, units=3072, activation=tf.nn.relu)\n",
    "            hidden10 = dense\n",
    "\n",
    "        with tf.variable_scope('digit_length'):\n",
    "            dense = tf.layers.dense(hidden10, units=7)\n",
    "            length = dense\n",
    "\n",
    "        with tf.variable_scope('digit1'):\n",
    "            dense = tf.layers.dense(hidden10, units=11)\n",
    "            digit1 = dense\n",
    "\n",
    "        with tf.variable_scope('digit2'):\n",
    "            dense = tf.layers.dense(hidden10, units=11)\n",
    "            digit2 = dense\n",
    "\n",
    "        with tf.variable_scope('digit3'):\n",
    "            dense = tf.layers.dense(hidden10, units=11)\n",
    "            digit3 = dense\n",
    "\n",
    "        with tf.variable_scope('digit4'):\n",
    "            dense = tf.layers.dense(hidden10, units=11)\n",
    "            digit4 = dense\n",
    "\n",
    "        with tf.variable_scope('digit5'):\n",
    "            dense = tf.layers.dense(hidden10, units=11)\n",
    "            digit5 = dense\n",
    "\n",
    "        length_logits, digits_logits = length, tf.stack([digit1, digit2, digit3, digit4, digit5], axis=1)\n",
    "        return length_logits, digits_logits\n",
    "\n",
    "    @staticmethod\n",
    "    def loss(length_logits, digits_logits, length_labels, digits_labels):\n",
    "        length_cross_entropy = tf.reduce_mean(tf.losses.sparse_softmax_cross_entropy(labels=length_labels, logits=length_logits))\n",
    "        digit1_cross_entropy = tf.reduce_mean(tf.losses.sparse_softmax_cross_entropy(labels=digits_labels[:, 0], logits=digits_logits[:, 0, :]))\n",
    "        digit2_cross_entropy = tf.reduce_mean(tf.losses.sparse_softmax_cross_entropy(labels=digits_labels[:, 1], logits=digits_logits[:, 1, :]))\n",
    "        digit3_cross_entropy = tf.reduce_mean(tf.losses.sparse_softmax_cross_entropy(labels=digits_labels[:, 2], logits=digits_logits[:, 2, :]))\n",
    "        digit4_cross_entropy = tf.reduce_mean(tf.losses.sparse_softmax_cross_entropy(labels=digits_labels[:, 3], logits=digits_logits[:, 3, :]))\n",
    "        digit5_cross_entropy = tf.reduce_mean(tf.losses.sparse_softmax_cross_entropy(labels=digits_labels[:, 4], logits=digits_logits[:, 4, :]))\n",
    "        loss = length_cross_entropy + digit1_cross_entropy + digit2_cross_entropy + digit3_cross_entropy + digit4_cross_entropy + digit5_cross_entropy\n",
    "        return loss\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "提取数据集中的样本数量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Meta(object):\n",
    "    def __init__(self):\n",
    "        self.num_train_examples = None\n",
    "        self.num_val_examples = None\n",
    "        self.num_test_examples = None\n",
    "\n",
    "    def save(self, path_to_json_file):\n",
    "        with open(path_to_json_file, 'w') as f:\n",
    "            content = {\n",
    "                'num_examples': {\n",
    "                    'train': self.num_train_examples,\n",
    "                    'val': self.num_val_examples,\n",
    "                    'test': self.num_test_examples\n",
    "                }\n",
    "            }\n",
    "            json.dump(content, f)\n",
    "\n",
    "    def load(self, path_to_json_file):\n",
    "        with open(path_to_json_file, 'r') as f:\n",
    "            content = json.load(f)\n",
    "            self.num_train_examples = content['num_examples']['train']\n",
    "            self.num_val_examples = content['num_examples']['val']\n",
    "            self.num_test_examples = content['num_examples']['test']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算模型的accuracy等"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Evaluator(object):\n",
    "    def __init__(self, path_to_eval_log_dir):\n",
    "        self.summary_writer = tf.summary.FileWriter(path_to_eval_log_dir)\n",
    "\n",
    "    def evaluate(self, path_to_checkpoint, path_to_tfrecords_file, num_examples, global_step):\n",
    "        batch_size = 128\n",
    "        num_batches = num_examples // batch_size\n",
    "        needs_include_length = False\n",
    "\n",
    "        with tf.Graph().as_default():\n",
    "            image_batch, length_batch, digits_batch = Donkey.build_batch(path_to_tfrecords_file,\n",
    "                                                                         num_examples=num_examples,\n",
    "                                                                         batch_size=batch_size,\n",
    "                                                                         shuffled=False)\n",
    "            length_logits, digits_logits = Model.inference(image_batch, drop_rate=0.0)\n",
    "            length_predictions = tf.argmax(length_logits, axis=1)\n",
    "            digits_predictions = tf.argmax(digits_logits, axis=2)\n",
    "\n",
    "            if needs_include_length:\n",
    "                labels = tf.concat([tf.reshape(length_batch, [-1, 1]), digits_batch], axis=1)\n",
    "                predictions = tf.concat([tf.reshape(length_predictions, [-1, 1]), digits_predictions], axis=1)\n",
    "            else:\n",
    "                labels = digits_batch\n",
    "                predictions = digits_predictions\n",
    "\n",
    "            labels_string = tf.reduce_join(tf.as_string(labels), axis=1)\n",
    "            predictions_string = tf.reduce_join(tf.as_string(predictions), axis=1)\n",
    "\n",
    "            accuracy, update_accuracy = tf.metrics.accuracy(\n",
    "                labels=labels_string,\n",
    "                predictions=predictions_string\n",
    "            )\n",
    "\n",
    "            tf.summary.image('image', image_batch)\n",
    "            tf.summary.scalar('accuracy', accuracy)\n",
    "            tf.summary.histogram('variables',\n",
    "                                 tf.concat([tf.reshape(var, [-1]) for var in tf.trainable_variables()], axis=0))\n",
    "            summary = tf.summary.merge_all()\n",
    "\n",
    "            with tf.Session() as sess:\n",
    "                sess.run([tf.global_variables_initializer(), tf.local_variables_initializer()])\n",
    "                coord = tf.train.Coordinator()\n",
    "                threads = tf.train.start_queue_runners(sess=sess, coord=coord)\n",
    "\n",
    "                restorer = tf.train.Saver()\n",
    "                restorer.restore(sess, path_to_checkpoint)\n",
    "\n",
    "                for _ in xrange(num_batches):\n",
    "                    sess.run(update_accuracy)\n",
    "\n",
    "                accuracy_val, summary_val = sess.run([accuracy, summary])\n",
    "                self.summary_writer.add_summary(summary_val, global_step=global_step)\n",
    "\n",
    "                coord.request_stop()\n",
    "                coord.join(threads)\n",
    "\n",
    "        return accuracy_val\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def _train(path_to_train_tfrecords_file, num_train_examples, path_to_val_tfrecords_file, num_val_examples,\n",
    "           path_to_train_log_dir, path_to_restore_checkpoint_file, training_options):\n",
    "    batch_size = training_options['batch_size']\n",
    "    initial_patience = training_options['patience']\n",
    "    num_steps_to_show_loss = 100\n",
    "    num_steps_to_check = 1000\n",
    "\n",
    "    with tf.Graph().as_default():\n",
    "        image_batch, length_batch, digits_batch = Donkey.build_batch(path_to_train_tfrecords_file,\n",
    "                                                                     num_examples=num_train_examples,\n",
    "                                                                     batch_size=batch_size,\n",
    "                                                                     shuffled=True)\n",
    "        length_logtis, digits_logits = Model.inference(image_batch, drop_rate=0.2)\n",
    "        loss = Model.loss(length_logtis, digits_logits, length_batch, digits_batch)\n",
    "\n",
    "        global_step = tf.Variable(0, name='global_step', trainable=False)\n",
    "        learning_rate = tf.train.exponential_decay(training_options['learning_rate'], global_step=global_step,\n",
    "                                                   decay_steps=training_options['decay_steps'], decay_rate=training_options['decay_rate'], staircase=True)\n",
    "        optimizer = tf.train.GradientDescentOptimizer(learning_rate)\n",
    "        train_op = optimizer.minimize(loss, global_step=global_step)\n",
    "\n",
    "        tf.summary.image('image', image_batch)\n",
    "        tf.summary.scalar('loss', loss)\n",
    "        tf.summary.scalar('learning_rate', learning_rate)\n",
    "        summary = tf.summary.merge_all()\n",
    "\n",
    "        config = tf.ConfigProto(allow_soft_placement = True)\n",
    "        config.gpu_options.per_process_gpu_memory_fraction = 0.45\n",
    "        config.gpu_options.allow_growth = True\n",
    "        with tf.Session(config = config) as sess:\n",
    "            summary_writer = tf.summary.FileWriter(path_to_train_log_dir, sess.graph)\n",
    "            evaluator = Evaluator(os.path.join(path_to_train_log_dir, 'eval/val'))\n",
    "\n",
    "            sess.run(tf.global_variables_initializer())\n",
    "            coord = tf.train.Coordinator()\n",
    "            threads = tf.train.start_queue_runners(sess=sess, coord=coord)\n",
    "\n",
    "            saver = tf.train.Saver()\n",
    "            if path_to_restore_checkpoint_file is not None:\n",
    "                assert tf.train.checkpoint_exists(path_to_restore_checkpoint_file), \\\n",
    "                    '%s not found' % path_to_restore_checkpoint_file\n",
    "                saver.restore(sess, path_to_restore_checkpoint_file)\n",
    "                print ('Model restored from file: %s' % path_to_restore_checkpoint_file)\n",
    "\n",
    "            print ('Start training')\n",
    "            patience = initial_patience\n",
    "            best_accuracy = 0.0\n",
    "            duration = 0.0\n",
    "\n",
    "            while True:\n",
    "                start_time = time.time()\n",
    "                _, loss_val, summary_val, global_step_val, learning_rate_val = sess.run([train_op, loss, summary, global_step, learning_rate])\n",
    "                duration += time.time() - start_time\n",
    "\n",
    "                if global_step_val % num_steps_to_show_loss == 0:\n",
    "                    examples_per_sec = batch_size * num_steps_to_show_loss / duration\n",
    "                    duration = 0.0\n",
    "                    print ('=> %s: step %d, loss = %f (%.1f examples/sec)' % (\n",
    "                        datetime.now(), global_step_val, loss_val, examples_per_sec))\n",
    "\n",
    "                if global_step_val % num_steps_to_check != 0:\n",
    "                    continue\n",
    "\n",
    "                summary_writer.add_summary(summary_val, global_step=global_step_val)\n",
    "\n",
    "                print ('=> Evaluating on validation dataset...')\n",
    "                path_to_latest_checkpoint_file = saver.save(sess, os.path.join(path_to_train_log_dir, 'latest.ckpt'))\n",
    "                accuracy = evaluator.evaluate(path_to_latest_checkpoint_file, path_to_val_tfrecords_file,\n",
    "                                              num_val_examples,\n",
    "                                              global_step_val)\n",
    "                print ('==> accuracy = %f, best accuracy %f' % (accuracy, best_accuracy))\n",
    "\n",
    "                if accuracy > best_accuracy:\n",
    "                    path_to_checkpoint_file = saver.save(sess, os.path.join(path_to_train_log_dir, 'model.ckpt'),\n",
    "                                                         global_step=global_step_val)\n",
    "                    print ('=> Model saved to file: %s' % path_to_checkpoint_file)\n",
    "                    patience = initial_patience\n",
    "                    best_accuracy = accuracy\n",
    "                else:\n",
    "                    patience -= 1\n",
    "\n",
    "                print ('=> patience = %d' % patience)\n",
    "                if patience == 0:\n",
    "                    break\n",
    "\n",
    "            coord.request_stop()\n",
    "            coord.join(threads)\n",
    "            print ('Finished')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "主函数，其中设置模型超参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def main(_):\n",
    "    data_dir = './data'\n",
    "    path_to_train_tfrecords_file = os.path.join(data_dir, 'train.tfrecords')\n",
    "    path_to_val_tfrecords_file = os.path.join(data_dir, 'val.tfrecords')\n",
    "    path_to_tfrecords_meta_file = os.path.join(data_dir, 'meta.json')\n",
    "    path_to_train_log_dir = './logs/train'\n",
    "    path_to_restore_checkpoint_file = None\n",
    "    training_options = {\n",
    "        'batch_size': 32,\n",
    "        'learning_rate': 1e-2,\n",
    "        'patience': 100,\n",
    "        'decay_steps': 10000,\n",
    "        'decay_rate': 0.9\n",
    "    }\n",
    "    meta = Meta()\n",
    "    meta.load(path_to_tfrecords_meta_file)\n",
    "    \n",
    "    _train(path_to_train_tfrecords_file, meta.num_train_examples,\n",
    "           path_to_val_tfrecords_file, meta.num_val_examples,\n",
    "           path_to_train_log_dir, path_to_restore_checkpoint_file,\n",
    "           training_options)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start training\n"
     ]
    }
   ],
   "source": [
    "tf.app.run(main=main)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
